Overspend? Late? Failure? What the Data Say About IT Project Risk in the Public Sector

Implementing large-scale information and communication technology (IT) projects carries large risks and easily might disrupt operations, waste taxpayers' money, and create negative publicity. Because of the high risks it is important that government leaders manage the attendant risks. We analysed a sample of 1,355 public sector IT projects. The sample included large-scale projects, on average the actual expenditure was $130 million and the average duration was 35 months. Our findings showed that the typical project had no cost overruns and took on average 24% longer than initially expected. However, comparing the risk distribution with the normative model of a thin-tailed distribution, projects' actual costs should fall within -30% and +25% of the budget in nearly 99 out of 100 projects. The data showed, however, that a staggering 18% of all projects are outliers with cost overruns >25%. Tests showed that the risk of outliers is even higher for standard software (24%) as well as in certain project types, e.g., data management (41%), office management (23%), eGovernment (21%) and management information systems (20%). Analysis showed also that projects duration adds risk: every additional year of project duration increases the average cost risk by 4.2 percentage points. Lastly, we suggest four solutions that public sector organization can take: (1) benchmark your organization to know where you are, (2) de-bias your IT project decision-making, (3) reduce the complexities of your IT projects, and (4) develop Masterbuilders to learn from the best in the field.Comment: Published in Commonwealth Secretariat (Eds.): Commonwealth Governance Handbook 2012/13: Democracy, development and public administration, London: Commonwealth Secretariat, December 2012. ISBN 978-1-908609-04-

Double Whammy - How ICT Projects are Fooled by Randomness and Screwed by Political Intent

The cost-benefit analysis formulates the holy trinity of objectives of project management - cost, schedule, and benefits. As our previous research has shown, ICT projects deviate from their initial cost estimate by more than 10% in 8 out of 10 cases. Academic research has argued that Optimism Bias and Black Swan Blindness cause forecasts to fall short of actual costs. Firstly, optimism bias has been linked to effects of deception and delusion, which is caused by taking the inside-view and ignoring distributional information when making decisions. Secondly, we argued before that Black Swan Blindness makes decision-makers ignore outlying events even if decisions and judgements are based on the outside view. Using a sample of 1,471 ICT projects with a total value of USD 241 billion - we answer the question: Can we show the different effects of Normal Performance, Delusion, and Deception? We calculated the cumulative distribution function (CDF) of (actual-forecast)/forecast. Our results show that the CDF changes at two tipping points - the first one transforms an exponential function into a Gaussian bell curve. The second tipping point transforms the bell curve into a power law distribution with the power of 2. We argue that these results show that project performance up to the first tipping point is politically motivated and project performance above the second tipping point indicates that project managers and decision-makers are fooled by random outliers, because they are blind to thick tails. We then show that Black Swan ICT projects are a significant source of uncertainty to an organisation and that management needs to be aware of

ICT project risk as pollution belief - A comparative essay in cultural theory

This essay briefly shines light on the current, normative, and positivist philosophies of how risk is debated in the field of project and program management. Then an alternative view is offered - Social Theories of Risk. This essay then analyses the National Programme for IT in the NHS (NPfIT), in its time the largest civilian ICT project in the world. The analysis focuses on the political struggles surrounding the project and asks the questions; What order was disturbed? Who disturbed the order? Who was blamed? and How was order re-established? The essay then contrasts notions from Social Theories of Risk with the idea of Governmentality to manage ICT project risk. The essay shows that Social Theories of Risk and Governmentality, both concepts not yet embraced in the ICT community, offer valuable insight into the study and the practice of ICT projects as clumsy solutions to wicked problems

Double whammy – how ICT projects are fooled by randomness and screwed by political intent

The Iron Triangle formulates the holy trinity of objectives of project management – cost, schedule, and benefits. As our previous research has shown, ICT projects deviate from their initial cost estimate by more than 10% in 8 out of 10 cases. Academic research has argued that Optimism Bias and Black Swan Blindness cause forecasts to fall short of actual costs. Firstly, optimism bias has been linked to effects of deception and delusion, which is caused by taking the inside-view and ignoring distributional information when making decisions. Secondly, we argued before that Black Swan Blindness makes decision-makers ignore outlying events even if decisions and judgements are based on the outside view. Using a sample of 1,471 ICT projects with a total value of USD 241 billion – we answer the question: Can we show the different effects of Normal Performance, Delusion, and Deception? We calculated the cumulative distribution function (CDF) of (actual-forecast)⁄forecast. Our results show that the CDF changes at two tipping points – the first one transforms an exponential function into a Gaussian bell curve. The second tipping point transforms the bell curve into a power law distribution with the power of 2. We argue that these results show that project performance up to the first tipping point is politically motivated and project performance above the second tipping point indicates that project managers and decision-makers are fooled by random outliers, because they are blind to thick tails. We then show that Black Swan ICT projects are a significant source of uncertainty to an organisation and that management needs to be aware of. Finally, we draw implications about the underlying generative processes that lead to power law behaviour, which might help to further understand the pitfalls and shortcomings of cost and cost risk management in ICT projects

Should we build more large dams? The actual costs of hydropower megaproject development

A brisk building boom of hydropower mega-dams is underway from China to Brazil. Whether benefits of new dams will outweigh costs remains unresolved despite contentious debates. We investigate this question with the "outside view" or "reference class forecasting" based on literature on decision-making under uncertainty in psychology. We find overwhelming evidence that budgets are systematically biased below actual costs of large hydropower dams - excluding inflation, substantial debt servicing, environmental, and social costs. Using the largest and most reliable reference data of its kind and multilevel statistical techniques applied to large dams for the first time, we were successful in fitting parsimonious models to predict cost and schedule overruns. The outside view suggests that in most countries large hydropower dams will be too costly in absolute terms and take too long to build to deliver a positive risk-adjusted return unless suitable risk management measures outlined in this paper can be affordably provided. Policymakers, particularly in developing countries, are advised to prefer agile energy alternatives that can be built over shorter time horizons to energy megaprojects

Does Infrastructure Investment Lead to Economic Growth or Economic Fragility? Evidence from China

The prevalent view in the economics literature is that a high level of infrastructure investment is a precursor to economic growth. China is especially held up as a model to emulate. Based on the largest dataset of its kind, this paper punctures the twin myths that, first, infrastructure creates economic value, and, second, China has a distinct advantage in its delivery. Far from being an engine of economic growth, the typical infrastructure investment fails to deliver a positive risk adjusted return. Moreover, China's track record in delivering infrastructure is no better than that of rich democracies. Where investments are debt-financed, overinvesting in unproductive projects results in the buildup of debt, monetary expansion, instability in financial markets, and economic fragility, exactly as we see in China today. We conclude that poorly managed infrastructure investments are a main explanation of surfacing economic and financial problems in China. We predict that, unless China shifts to a lower level of higher-quality infrastructure investments, the country is headed for an infrastructure-led national financial and economic crisis, which is likely also to be a crisis for the international economy. China's infrastructure investment model is not one to follow for other countries but one to avoid

Report for the Edinburgh Tram Inquiry

This report reviews the Edinburgh tram project's risk management. Projects frequently overrun their cost and timelines and fall short on intended benefits. Cost, schedule, and benefit risk of projects need to be carefully considered to avoid this. The report describes and evaluates risk assessment and management for the Edinburgh tram. The report was produced as part of the Edinburgh Tram Inquiry. Keywords: risk assessment, risk management, infrastructure, megaprojects, optimism bias, strategic misrepresentation, planning fallacy, behavioral science

Regression to the Tail: Why the Olympics Blow Up

The Olympic Games are the largest, highest-profile, and most expensive megaevent hosted by cities and nations. Average sports-related costs of hosting are $12.0 billion. Non-sports-related costs are typically several times that. Every Olympics since 1960 has run over budget, at an average of 172 percent in real terms, the highest overrun on record for any type of megaproject. The paper tests theoretical statistical distributions against empirical data for the costs of the Games, in order to explain the cost risks faced by host cities and nations. It is documented, for the first time, that cost and cost overrun for the Games follow a power-law distribution. Olympic costs are subject to infinite mean and variance, with dire consequences for predictability and planning. We name this phenomenon "regression to the tail": it is only a matter of time until a new extreme event occurs, with an overrun larger than the largest so far, and thus more disruptive and less plannable. The generative mechanism for the Olympic power law is identified as strong convexity prompted by six causal drivers: irreversibility, fixed deadlines, the Blank Check Syndrome, tight coupling, long planning horizons, and an Eternal Beginner Syndrome. The power law explains why the Games are so difficult to plan and manage successfully, and why cities and nations should think twice before bidding to host. Based on the power law, two heuristics are identified for better decision making on hosting. Finally, the paper develops measures for good practice in planning and managing the Games, including how to mitigate the extreme risks of the Olympic power law.Comment: arXiv admin note: text overlap with arXiv:1607.0448